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DETAILED ACTION 

1 . Claims 1-5, 8-16 and 20-21 are pending in this application. 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/22/09 
has been entered. 



Claim Rejections - 35 USC §112 

2. Claim 21 rejected under 35 U.S.C. 112, first paragraph, as failing to comply with 
the written description requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to reasonably convey to one skilled in 
the relevant art that the inventor(s), at the time the application was filed, had possession 
of the claimed invention. 

Claim 21 does not appear to accurately reflect the subject matter disclosed in the 
specification and does not operate properly with respect to what is explained in the 
specification. The probability appears to evolve from the frequency of the set of 
attributes causing a specific bin number, h, however Claim 21 attempts to claim that the 
probability is calculated from the equation stated. Therefore this is inconsistent with the 
specification, and the meaning of the claim is indeterminable and therefore has not 
been further treated on the merits. 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 1 03(a). 

5. Claims 1-6, 8-12,14-16 and 20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Snook, US Patent 6400378, and further in view of Dufaux, US Patent 
6711587. 

As in Claims 1 and 15, Snook teaches a media handling system in which 
candidate video sequences are displayed on a display screen in schematic form for 
selection by a user (Fig. 3 and corresponding text) and method (Fig. 3 and 
corresponding text), the system and method comprising: a display screen configured to 
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display representations of the candidate video sequences for selection by a user (ref. 
120), each representation including one or more images derived from the respective 
video sequences (Col. 2, lines 64-66); and a user control means for defining a set of 
one or more of the video sequences (Col. 2, line 55-Col. 3, line 1). While Snook 
teaches displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences, they fail to show 
the means for detecting human faces in the candidate video sequences to represent the 
candidate video sequences as a keyframe as recited in the claims. In the same field of 
the invention, Dufaux teaches a media handling system similar to that of Snook. In 
addition, Dufaux further teaches means for detecting human faces in the candidate 
video sequences (Fig. 5, ref. 500 and corresponding text); and assigning them as the 
key frame representations of the candidate video sequences (Col. 4, lines 4-7). It would 
have been obvious to one of ordinary skill in the art, having the teachings of Snook and 
Dufaux before him at the time the invention was made, to modify displaying video 
sequences, represented by a keyframe thumbnail, that may be for selected by a user to 
define a set of one or more video sequences taught by Snook to include the means for 
detecting human faces in the candidate video sequences and assigning them as the key 
frame representations of the candidate video sequences of Dufaux, in order to obtain 
means for detecting human faces in candidate video sequences, displaying video 
sequences, represented by an image representing human faces derived from the 
respective video sequences, that may be for selected by a user to define a set of one or 
more video sequences. One would have been motivated to make such a combination 
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because a way for the system to select a more relevant, user friendly representative 
image would have been obtained, as taught by Dufaux. 

Snook fails to explicitly teach detecting human faces derived from the respective 
video sequences and displaying the one with the highest probability of containing a face 
in the group. Dufaux teaches the detecting means is operable to detect a probability of 
a human face being present in each field or frame of the video sequences (Col. 1 , lines 
55-57); each displayed representation of a candidate video sequence including one or 
more images representing human faces which have the highest probability levels 
amongst the respective video sequences (Col. 12, line 4 et seq.). It would have been 
obvious to one of ordinary skill in the art, having the teachings of Snook and Dufaux 
before him at the time the invention was made, to modify displaying video sequences, 
represented by a keyframe thumbnail, that may be for selected by a user to define a set 
of one or more video sequences and a further ordered representation of a group of at 
least a subset of the video sequences forming the output media product taught by 
Snook to include the detecting human faces derived from the respective video 
sequences and displaying the one with the highest probability of containing a face in the 
group of Dufaux, in order to obtain a further ordered representation of a group of at least 
a subset of the video sequences forming the output media product, the video 
sequences represented by an image with the highest probability of containing human 
faces derived from the respective video sequences. One would have been motivated to 
make such a combination because a way for the system to select a more relevant, user 
friendly representative image would have been obtained, as taught by Dufaux. 
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Snook fails to explicitly teach weighting frames according to the size of detected 
human faces derived from the respective video sequences. Dufaux teaches the 
detecting means is operable to detect a probability of a human face being present in 
each field or frame of the video sequences (Fig. 5) and to weight at least some of the 
detected probability levels in dependence on the size of the detected face (Col. 12, lines 
4 et seq.). It would have been obvious to one of ordinary skill in the art, having the 
teachings of Snook and Dufaux before him at the time the invention was made, to 
modify displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences and a further 
ordered representation of a group of at least a subset of the video sequences forming 
the output media product taught by Snook to include the weighting frames according to 
the size of detected human faces derived from the respective video sequences of 
Dufaux, in order to obtain a further ordered representation of a group of at least a 
subset of the video sequences forming the output media product, the video sequences 
represented by an image with the highest probability of containing a particular sized 
human faces derived from the respective video sequences. One would have been 
motivated to make such a combination because a way for the system to select a more 
accurate face detection method for keyframe selection would have been obtained, as 
taught by Dufaux. 

As in Claim 2, Snook teaches the set of one or more of the video sequences is 
an ordered edited set forming an output media product (Col. 2, lines 62-64). 
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As in Claim 3, the above combination of Snook and Dufaux teach the system of 
Claim 1 as rejected supra. Snook further teaches a further ordered representation of a 
group of at least a subset of the video sequences forming the output media product 
(Snook, Col. 8, lines 2-3). Snook fails to explicitly teach images representing human 
faces derived from the respective video sequences in the group. Dufaux teaches 
detected human faces represent their respective video sequences (Dufaux, Col. 4, lines 
4-7). It would have been obvious to one of ordinary skill in the art, having the teachings 
of Snook and Dufaux before him at the time the invention was made, to modify 
displaying video sequences, represented by a keyframe thumbnail, that may be for 
selected by a user to define a set of one or more video sequences and a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product taught by Snook to include the means for detecting human faces in the 
candidate video sequences and assigning them as the key frame representations of the 
candidate video sequences of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product, the video sequences represented by an image representing human 
faces derived from the respective video sequences. One would have been motivated to 
make such a combination because a way for the system to select a more relevant, user 
friendly representative image would have been obtained, as taught by Dufaux. 

As in Claim 4, Snook teaches the ordered representation is a timeline 
representation ("video timeline", Col. 6, line 29), providing an ordered representation of 
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the group of video sequences forming the output media product along a generally 
rectilinear path on the display screen (Fig. 6A and corresponding text). 

As in Claim 5, Snook teaches which the ordered representation may be scaled 
so as to vary the proportion of the video sequences forming the output media product 
which are currently displayed in the ordered representation (Fig. 6A ref. 625 and 
corresponding text). 

As in Claim 6, Snook fails to explicitly teach detecting human faces derived from 
the respective video sequences and displaying the one with the highest probability of 
containing a face in the group. Dufaux teaches the detecting means is operable to 
detect a probability of a human face being present in each field or frame of the video 
sequences (Col. 1 , lines 55-57); each displayed representation of a candidate video 
sequence including one or more images representing human faces which have the 
highest probability levels amongst the respective video sequences (Col. 12, line 4 et 
seq.). It would have been obvious to one of ordinary skill in the art, having the teachings 
of Snook and Dufaux before him at the time the invention was made, to modify 
displaying video sequences, represented by a keyframe thumbnail, that may be for 
selected by a user to define a set of one or more video sequences and a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product taught by Snook to include the detecting human faces derived from the 
respective video sequences and displaying the one with the highest probability of 
containing a face in the group of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
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media product, the video sequences represented by an image with the highest 
probability of containing human faces derived from the respective video sequences. 
One would have been motivated to make such a combination because a way for the 
system to select a more relevant, user friendly representative image would have been 
obtained, as taught by Dufaux. 

As in Claim 8, Snook fails to explicitly teach weight the probability levels so that 
detected faces closer in size to a desired representation size are more likely to be 
selected to form a displayed representation. Dufaux teaches the detecting means is 
operable to detect a probability of a human face being present in each field or frame of 
the video sequences (Fig. 5) and to weight at least some of the detected probability 
levels in dependence on the size of the detected face (Col. 12, lines 4 et seq.); each 
displayed representation of a candidate video sequence including one or more images 
representing human faces which have the highest weighted probability levels amongst 
the respective video sequences (Col. 12, line 4 et seq.). Dufaux further teaches the 
detecting means weights the probability levels so that detected faces closer in size to a 
desired representation size are more likely to be selected to form a displayed 
representation (Col. 2, lines 12-25). It would have been obvious to one of ordinary skill 
in the art, having the teachings of Snook and Dufaux before him at the time the 
invention was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences and a further ordered representation of a group of at least a subset of the 



Application/Control Number: 10/536,814 Page 10 

Art Unit: 2179 

video sequences forming the output media product taught by Snook to include the 
weight the probability levels so that detected faces closer in size to a desired 
representation size are more likely to be selected to form a displayed representation 
Dufaux, in order to obtain a further ordered representation of a group of at least a 
subset of the video sequences forming the output media product, the video sequences 
represented by an keyframe image determined according to weight, weight the 
probability levels so that detected faces closer in size to a desired representation size 
are more likely to be selected to form a displayed representation. One would have been 
motivated to make such a combination because a way for the system to select a more 
accurate facial detection method for keyframe selection would have been obtained, as 
taught by Dufaux. 

As in Claim 9, Snook fails to explicitly teach the detecting means applies the 
weighting over a subset of the fields or frames of a video sequence. Dufaux teaches 
the detecting means applies the weighting over a subset of the fields or frames of a 
video sequence (Col. 2, lines 20-24). It would have been obvious to one of ordinary skill 
in the art, having the teachings of Snook and Dufaux before him at the time the 
invention was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences and a further ordered representation of a group of at least a subset of the 
video sequences forming the output media product taught by Snook to include the 
weighting frames according to the size of detected human faces derived from the 
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respective video sequences and displaying the one with the highest probability of 
containing a face in the group of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product, the video sequences represented by a detected keyframe image, the 
detecting means is operable to apply the weighting over a subset of the fields or frames 
of a video sequence. One would have been motivated to make such a combination 
because a way for the system to select a more accurate face detection method for 
keyframe selection would have been obtained, as taught by Dufaux. 

As in Claims 10-11, Snook fails to explicitly teach the weighting over frames of a 
video sequence. Dufaux teaches the detecting means applies the weighting over 
frames of a video sequence (Col. 2, lines 20-24). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Snook and Dufaux before him at the 
time the invention was made, to modify displaying video sequences, represented by a 
keyframe thumbnail, that may be for selected by a user to define a set of one or more 
video sequences and a further ordered representation of a group of at least a subset of 
the video sequences forming the output media product taught by Snook to include the 
detecting means applies the weighting over frames of a video sequence of Dufaux, in 
order to obtain a further ordered representation of a group of at least a subset of the 
video sequences forming the output media product, the video sequences represented 
by a detected keyframe image, the detecting means applies the weighting over frames 
of a video sequence. One would have been motivated to make such a combination 
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because a way for the system to select a more accurate face detection method for 
keyframe selection would have been obtained, as taught by Dufaux. 

As in Claim 12, Snook teaches selection of a displayed representation by the 
user control causes the display of the corresponding video sequence (Col. 3, lines 33- 
37). 

As in Claim 14, Snook fails to explicitly teach indicating that faces detected in two 
or more respective ones of the candidate video sequences represent the same person's 
face. Dufaux teaches a user control means indicates that faces detected in two or more 
of the candidate video sequences represent the same person's face (Fig. 5 and 
corresponding text). It would have been obvious to one of ordinary skill in the art, 
having the teachings of Snook and Dufaux before him at the time the invention was 
made, to modify displaying video sequences, represented by a keyframe thumbnail, that 
may be for selected by a user to define a set of one or more video sequences and a 
further ordered representation of a group of at least a subset of the video sequences 
forming the output media product taught by Snook to include the indicating that faces 
detected in two or more respective ones of the candidate video sequences represent 
the same person's face of Dufaux, in order to obtain a further ordered representation of 
a group of at least a subset of the video sequences forming the output media product, 
the video sequences represented by a detected keyframe image with detected faces 
and indicating that faces detected in two or more respective ones of the candidate video 
sequences represent the same person's face. One would have been motivated to make 
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such a combination because a way for the system to select a more accurate face 
detection method for keyframe selection would have been obtained, as taught by 
Dufaux. 

As in Claim 16, Snook teaches computer readable storage medium (Fig. 2 and 
corresponding text) encoded with a computer readable program configured to cause an 
information processing apparatus to execute the method of Claim 15 (See Claim 15 
rejection supra). 

As in Claim 20, Snook teaches a media handling system in which candidate 
video sequences are displayed on a display screen in schematic form for selection by a 
user (Fig. 3 and corresponding text), the system comprising: a display screen 
configured to display representations of the candidate video sequences for selection by 
a user (ref. 120), each representation including one or more images derived from the 
respective video sequences (Col. 2, lines 64-66); and a user control for defining a set of 
one or more of the video sequences (Col. 2, line 55-Col. 3, line 1). While Snook 
teaches displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences, they fail to show 
the detector to detect human faces in the candidate video sequences to represent the 
candidate video sequences as a keyframe as recited in the claims. In the same field of 
the invention, Dufaux teaches a media handling system similar to that of Snook. In 
addition, Dufaux further teaches detector to detect human faces in the candidate video 



Application/Control Number: 10/536,814 Page 14 

Art Unit: 2179 

sequences (Fig. 5, ref. 500 and corresponding text); and assigning them as the key 
frame representations of the candidate video sequences (Col. 4, lines 4-7). It would 
have been obvious to one of ordinary skill in the art, having the teachings of Snook and 
Dufaux before him at the time the invention was made, to modify displaying video 
sequences, represented by a keyframe thumbnail, that may be for selected by a user to 
define a set of one or more video sequences taught by Snook to include the detector to 
detect human faces in the candidate video sequences and assigning them as the key 
frame representations of the candidate video sequences of Dufaux, in order to obtain a 
detector to detect human faces in candidate video sequences, displaying video 
sequences, represented by an image representing human faces derived from the 
respective video sequences, that may be for selected by a user to define a set of one or 
more video sequences. One would have been motivated to make such a combination 
because a way for the system to select a more relevant, user friendly representative 
image would have been obtained, as taught by Dufaux. 

Snook fails to explicitly teach detecting human faces derived from the respective 
video sequences and displaying the one with the highest probability of containing a face 
in the group. Dufaux teaches the detecting means is operable to detect a probability of 
a human face being present in each field or frame of the video sequences (Col. 1 , lines 
55-57); each displayed representation of a candidate video sequence including one or 
more images representing human faces which have the highest probability levels 
amongst the respective video sequences (Col. 12, line 4 et seq.). It would have been 
obvious to one of ordinary skill in the art, having the teachings of Snook and Dufaux 
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before him at the time the invention was made, to modify displaying video sequences, 
represented by a keyframe thumbnail, that may be for selected by a user to define a set 
of one or more video sequences and a further ordered representation of a group of at 
least a subset of the video sequences forming the output media product taught by 
Snook to include the detecting human faces derived from the respective video 
sequences and displaying the one with the highest probability of containing a face in the 
group of Dufaux, in order to obtain a further ordered representation of a group of at least 
a subset of the video sequences forming the output media product, the video 
sequences represented by an image with the highest probability of containing human 
faces derived from the respective video sequences. One would have been motivated to 
make such a combination because a way for the system to select a more relevant, user 
friendly representative image would have been obtained, as taught by Dufaux. 

Snook fails to explicitly teach weighting frames according to the size of detected 
human faces derived from the respective video sequences. Dufaux teaches the 
detecting means is operable to detect a probability of a human face being present in 
each field or frame of the video sequences (Fig. 5) and to weight at least some of the 
detected probability levels in dependence on the size of the detected face (Col. 12, lines 
4 et seq.). It would have been obvious to one of ordinary skill in the art, having the 
teachings of Snook and Dufaux before him at the time the invention was made, to 
modify displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences and a further 
ordered representation of a group of at least a subset of the video sequences forming 
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the output media product taught by Snook to include the weighting frames according to 
the size of detected human faces derived from the respective video sequences of 
Dufaux, in order to obtain a further ordered representation of a group of at least a 
subset of the video sequences forming the output media product, the video sequences 
represented by an image with the highest probability of containing a particular sized 
human faces derived from the respective video sequences. One would have been 
motivated to make such a combination because a way for the system to select a more 
accurate face detection method for keyframe selection would have been obtained, as 
taught by Dufaux. 

6. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Snook, 
US Patent 6400378, Dufaux, US Patent 671 1587 and further in view of Trivedi et al., US 
Patent Application Publication 2006/0187305, hereinafter Trivedi. 

Snook and Dufaux teach means for detecting human faces in candidate video 
sequences, displaying video sequences, represented by an image representing human 
faces derived from the respective video sequences, that may be for selected by a user 
to define a set of one or more video sequences (Claim 1 rejection supra). While Snook 
and Dufaux teach displaying video sequences, represented by a keyframe thumbnail, 
that may be for selected by a user to define a set of one or more video sequences, 
detecting human faces in the candidate video sequences to represent the candidate 
video sequences as a keyframe, they fail to show the candidate video sequences are 
selected from a video sequence captured by a surveillance camera as recited in the 
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claims. In the same field of the invention, Trivedi teaches a media handling system and 
facial detection similar to that of Snook and Dufaux. In addition, Trivedi further teaches 
in which the candidate video sequences are selected from a video sequence captured 
by a surveillance camera (Par. 0038). It would have been obvious to one of ordinary 
skill in the art, having the teachings of Snook, Dufaux before him at the time the 
invention was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences, detecting human faces in the candidate video sequences and assigning 
them as the key frame representations of the candidate video sequences taught by 
Snook and Dufaux to include the candidate video sequences are selected from a video 
sequence captured by a surveillance camera of Trivedi, in order to obtain means for 
detecting human faces in candidate video sequences captured by a surveillance 
camera, displaying video sequences, represented by an image representing human 
faces derived from the respective video sequences, that may be for selected by a user 
to define a set of one or more video sequences. One would have been motivated to 
make such a combination because a way for police and security guards to single out 
people from video surveillance would have been obtained, as taught by Trivedi. 

Response to Arguments 

Applicant's arguments filed 5/22/09 have been fully considered but they are not 
persuasive. 
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The applicant states "Snook and Dufaux do not disclose or suggest "a method of 
media handling in which candidate video sequences are displayed on a display screen 
in schematic form for selection by a user". The examiner disagrees and further notes 
that this is a part of the preamble and is not disclosed as a limitation in the body of the 
claim. The stated preamble is not given patentable weight as it fails to breathe life, 
meaning, and vitality into the claims. 

In response to applicant's argument that the examiner's conclusion of 
obviousness is based upon improper hindsight reasoning, the examiner disagrees. As 
stated in the advisory, it must be recognized that any judgment on obviousness is in a 
sense necessarily a reconstruction based upon hindsight reasoning. But so long as it 
takes into account only knowledge which was within the level of ordinary skill at the time 
the claimed invention was made, and does not include knowledge gleaned only from the 
applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 
1 392, 1 70 USPQ 209 (CCPA 1 971 ). One of ordinary skill would be able to see the 
advantage of using skin pixels to weight more desirable frames by incorporating this into 
the invention of Snook which displays a keyframe representing a video sequence for 
selection. 

In response to the applicant's argument that "Dufaux is silent regarding weighting 
a detected probability level depending on a size of a detected face" (page 10, lines 21- 
22) the examiner disagrees. Dufaux teaches giving weight, referred to as a weighting 
factor (Col. 2, line 24) to determine the shot score, according to different limitations 
including the skin pixels (Col. 2, line 22). A larger face in a shot would give the shot a 
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higher number of skin pixels. This is further illustrated by Dufaux in Col. 6, beginning on 
line 41 , when they begin to discuss the process of determining if the frame has people 
in it. Frames that have smaller faces are not given as much weight because they "are 
not likely to be representative of the video" as a whole. 



The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1 .1 1 1 (c) to 
consider these references fully when responding to this action. The documents cited 
therein teach similar video editing and keyframe selection techniques as well as 
methods for detecting human faces within video. 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sara M. Hanne whose telephone number is (571) 272- 
4135. The examiner can normally be reached on M-F 7:30am-4:00pm, off on 
alternating Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, WEILUN LO can be reached on (571) 272-4847. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Sara M Hanne/ 
Examiner, Art Unit 2179 



